Python modules

This modules are provided with the library to ease the interoperability between I/O of the Fortran library with python scripting

Utils module

This file contains some low-level useful functions

class futile.Utils.Node(obj, valid_if=None)[source]

Bases: object

An object that is associated to a queue. Has a requirement and a validity function as well as a generation function that is triggered in case the node is not valid.


obj (object) – a generic object

requires(node, generator)[source]

Set the dependency between two nodes.

  • node (Node) – the node from which this one depends. If this is a valid node, and the present is not, it should employ the generator function for creating the node.

  • generator (func) – function which should be called to make the node valid. Should have as arguments the two objects associated the node (the current one is the first argument)

property valid

Set the function as callback to check the validity of the node.


func (func) – function that has the node object as a the first argument and returns a boolean which assess the validity of the function.


This function makes the node valid by validating its dependency and by calling the generator function if this is not the case.


the value of self.valid. Should be true.

Return type


class futile.Utils.ObjectSerialization(obj=None, files=None, version=None)[source]

Bases: object

Serialization of a class into an archive.

This class can be employed each time that we would like to identify the minimal set of files needed to instantiate an object. Files and objects are put in an archive that can in this way be used to retrieve and reinstantiate the object.

  • obj (object) – a instance of the object to be serailized. Necessary in case some attributes of it have to be included in the serialization.

  • version (str) – the version of the desired serialization. If absent, the default is considered.

  • files (dict) – dictionary of the files to be included. The dictionary should be of the form {<filename>: <abspath>} or rather {<filename>: {‘archive’:<archive_path>, ‘file’:<member>}} in which case the file <member> of the archive <archive_path> will be included. The file will be included in the serialization as <filename>.

cached_attributes = {'1.0': {}}
dump(archive, extra_encoder_functions=[])[source]

Create an archive with the entire set of information of the Serialization. Such a tarfile should be such that the same analysis of the object is possible

  • archive (str) – path of the archive to serialize the object to.

  • extra_encoder_functions (list) – see :py:func:serialize_objects.

classmethod load(archive, init_function, tmpdir_prefix='', load_functions={}, serialization_version=None, **kwargs)[source]

Create a class instance from a serialized archive.

  • archive (str) – the path of the archive

  • init_function (func) – function that should be called at the class instantiation

  • load_functions (dict) – dictionary of load functions per cached attribute. The function signature is the class instance, the files list, and the attribute.

  • serialization_version (str) – version of the load

  • tmpdir_prefix (str) – prefix to be added to the temporary directory

  • **kwargs – other arguments that have to be passed to the

  • init_function


Instance of the class

version = '1.0'

version of the serialization

futile.Utils.create_tarball(filename, files, objects={})[source]

Assemble files and objects in a tarball

  • filename (str) – the name of the archive. Determine the tarball compression method from its extension.

  • files (dict,set) – file paths that have to be included in the tarball. If it is a dict, it should be in the form “{arcname : file}”, where file is the path of the file to be put, and arcname is the name of thefile that would be used in the archive. If it is a set, the file will preserve its name in the archive

  • objects (dict) – dictionary ‘{arcname: buffer}’ of the buffers that will have to be serialized in the arcname position the buffers are given as class::io.BytesIO instance, following specification of the func:serialize_objects function.

futile.Utils.data_path(archive, dest='.', path='datalake', branch='main', github_repo='BigDFT-group/resources')[source]
futile.Utils.dict_get(inp, *subfields)[source]

Find the value of the provided sequence of keys in the dictionary, if available.

Retrieve the value of the dictionary in a sequence of keys if it is available. Otherwise it provides as default value the last item of the sequence subfields.

  • inp (dict) – the top-level dictionary. Unchanged on exit.

  • subfields (str,object) – keys, ordered by level, that have to be retrieved from topmost level of inp. The last item correspond to the value to be set.


The value provided by the sequence of subfields if available, otherwise the default value given as the last item of the subfields sequence.

futile.Utils.dict_merge(dest, src)[source]

Recursive dict merge. Inspired by dict.update(), instead of updating only top-level keys, dict_merge recurses down into dicts nested to an arbitrary depth, updating keys. The src is merged into dest. From angstwad/

  • dest (dict) – dict onto which the merge is executed

  • src (dict) – dict merged into dest

futile.Utils.dict_set(inp, *subfields)[source]

Ensure the provided fields and set the value

Provide a entry point to the dictionary. Useful to define a key in a dictionary that may not have the previous keys already defined.

  • inp (dict) – the top-level dictionary

  • subfields (str,object) – keys, ordered by level, that have to be retrieved from topmost level of inp. The last item correspond to the value to be set .


>>> inp={}
>>> dict_set(inp,'dft','nspin','mpol',2)
>>> print (inp)
{'dft': {'nspin': {'mpol': 2}}}
futile.Utils.ensure_copy(src, dest)[source]

Copy src into dest.

Guarantees that the file dest is a copy of the file src.

  • src (str) – path of the source file. Should be valid.

  • dest (src) – path of the destination file


True if the file needed to be copied, False if src

and dest are identical

Return type



Guarantees the existance on the directory given by the (relative) file_path


file_path (str) – path of the directory to be created


True if the directory needed to be created,

False if it existed already or if an error happened during the creation.

Return type


futile.Utils.ensure_object(filename, code_if_obj_not_found=None, glob=None, loc=None)[source]

Identify a pickle file to save a given object on it. In case this file is present, read the object from it. Otherwise, assume that the object is ready to be dumped and write it in the file.

  • filename (str) – the path of the file in which the object is saved/loaded.

  • code_if_obj_not_found (str) – the code to be evaluated if the object has

  • found. (not been) –

  • globals (dict) – the global variables dictionary

  • locals (dict) – the local variables dictionary


The result from the pickle file or from the code to be executed

Return type


futile.Utils.execute_code_if(condition, code, glob=None, loc=None)[source]

Execute code if condition is true

  • condition (bool) – if true the code is executed

  • code_if_obj_not_found (str) – the code to be evaluated if the object has not been found.

  • globals (dict) – the global variables dictionary

  • locals (dict) – the local variables dictionary


the object returned from the code executed, None otherwise

futile.Utils.file_list(directory='.', suffix=None, prefix=None, exclude=None, include_directory_path=False)[source]

Return the list of the files inside a given directory

  • directory (str) – path of the directory to search into

  • suffix (str) – the suffix that the files should have

  • prefix (str) – the prefix that the files should have

  • exclude (str) – exclude the files which matches this string from the list

  • include_directory_path (bool) – if True includes the path of the directory in the list.


list of the files that matches the requirements.

Return type



Determine the time of the last modification of a file.


filename (str) – path of the file to inspect.


time of the modified file. Returns 0 if the file does not exist.

Return type


futile.Utils.fill_dictionary_in_parallel(nthreads, keys, func, **kwargs)[source]

Fill a dictionary of a given set of keys with the return value of a function which accept this key as a first argument

  • nthreads (int) – the number of threads of the pool

  • keys (list) – the arguments of the function. Will be the list of the dictionary

  • func (func) – the python function that has the key as a last argument

  • **kwargs – further arguments of the function, if needed


the key-> obj dictionary with the obj the return value of func

Return type


futile.Utils.find_files(regexp, archive=None)[source]

Returns a list of the paths to the files that follow the regular expression regexp. They are searched from the current working directory or from an archive given as optional argument.

  • regexp (string) – A regular expression

  • archive – an opened tarfile archive (optional)


a list of all the paths that agree with the regexp

Return type

list of strings


ValueError if the regexp does not find a single path.


#Find all python files in the current working directory

#An exmple outside of the current working directory

#Example using a tarfile
import tarfile
my_archive ='archive.tar.gz')
find_files('*/*/log-*.yaml', archive=my_archive)

Useful to make float from strings compatible from fortran


scalar (str, float) – When string representing a float that might be given in fortran notation, otherwise it might be a floating point


float. The value associated to scalar as a floating point number


>>> # this would be the same with "1.e-4" or with 0.0001
>>> floatify('1.d-4')
futile.Utils.function_signature_regenerator(target_kwargs_function, fun_name='', fun_docstring='', **kwargs)[source]

Generate the function of the name provided by fun_name, with signature provided by the kwargs dictionary.

  • target_kwargs_function (func) – keyword arguments function that will be used for the generated function.

  • fun_name (str) – name of the regenerated function. If empty it will be the target_kwargs_functon.__name__ prefixed by regenerated, which will be copied in the docstring of the regenerated function.

  • fun_docstring (str) – docstring of the generated function, if empty it will take the docstring from target_kwargs_function.

  • **kwargs – keyword arguments which will represent the signature of the generated function.


>>> def write_kwargs(**kwargs):
>>>     """
>>>     Convert keyword arguments into a string
>>>     """
>>>     return str(kwargs)
>>> write_opts=function_signature_regenerator(write_kwargs,
>>>                                           fun_name='write_opts',
>>>                                           opt1='default1',
>>>                                           opt2='default2')
>>> help(write_opts)
>>> print (write_opts())
Help on function write_opts:
write_opts(opt1=’default1’, opt2=’default2’)

Convert keyword arguments into a string

{‘opt1’: ‘default1’, ‘opt2’: ‘default2’}

futile.Utils.get_curl_command(sha, size, repo)[source]
futile.Utils.kw_pop(*args, **kwargs)[source]

Treatment of kwargs. Eliminate from kwargs the tuple in args.


>>> kwargs = {'one': 1, 'two': 2, 'three': 3}
>>> # Existing value, default ignored
>>> kw2, two_maybe = kw_pop('two', 100, **kwargs)
>>> print (kw2, two_maybe)
{'one': 1, 'three': 3}, 2
>>> # Not, existing value, default considered
>>> kw2, four_maybe = kw_pop('four', 4, **kwargs)
>>> print (kw2, four_maybe)
{'one': 1, 'two': 2, 'three': 3}, 4

Transform the instance inp into a python dictionary. If inp is already a dictionary, it performs a copy.


inp (dict) – a instance of a Class which inherits from dict


the copy of the class, converted as a dictionary

Return type


futile.Utils.merge_two_dicts(x, y)[source]
futile.Utils.more_recent_than_parent(filename, parent)[source]

Filename should be more recent than parent


Control if the file has nonzero size and exists

futile.Utils.option_line_generator(separator='--', **kwargs)[source]

Associate to each of the keyword arguments a command line argument.

  • separator (str) – The string needed to separate the options.

  • arguments (Might be '--' for command-line) –

  • ' (but also) –

  • '

  • signatures. (for function) –


The separator comes before the first argument therefore pay attention to lstrip it in case you want to use it as a function signature string.


>>> option_line_generator(arg1='val1',arg2='val2')
'--arg1=val1 --arg2=val2'
futile.Utils.property_attribute(self, attribute, code_if_not_found)[source]
futile.Utils.push_path(inp, *keys)[source]

Follow in the dictionary inp the path indicated by the keys. If this path does not exists creates it.

  • inp (dict) – dictionary

  • keys (str) – keys of the path to follow


(branch,``key``) tuple, where

  • branch (dict): the dictionary of the second-last item of the path

  • key (str): the last item of the path


>>> inp={}
>>> d,key=push_path(inp,'dft','nspin','mpol')
>>> print (d,key)
>>> print (inp)
{'dft': {'nspin': {}}}
>>> inp={'dft': {'nspin': {'mpol': 2}}}
>>> d,key=push_path(inp,'dft','nspin','mpol')
>>> print (d,key)
>>> print (inp)
{'mpol': 2},'mpol'
{'dft': {'nspin': {'mpol': 2}}}
futile.Utils.serialize_objects(objects, extra_encoder_functions=[])[source]

Convert a dictionary of objects into buffers. Employs json serialization into StringIO instances

  • objects (dict) – dictionary of key/value pair of objects to be serialized

  • extra_encoder_functions (list)) – list of dictionaries of the format {‘cls’: Class, ‘func’: function} which is employed in the serialization


dictionary of key/buffer pairs

Return type


futile.Utils.sort_lists(sort_by, ascending, *lists)[source]

Sort lists altogether following the lists indicated by the sort_by index.

  • sort_by (int) – the index of the list which has to be taken as reference for sorting

  • ascending (bool) – Sort is performed in ascending order if True

  • *lists – sequence of lists to be mutually sorted. They have to be of the same length.


tuple of sorted lists

Example: >>> l1=[5,3,4] >>> l2=[‘c’,’t’,’q’] >>> l3=[6,3,7] >>> print (sort_lists(0,True,l1,l2,l3)) >>> print (sort_lists(2,True,l1,l2,l3)) [(3, 4, 5), (‘t’, ‘q’, ‘c’), (3, 7, 6)] [(3, 5, 4), (‘t’, ‘c’, ‘q’), (3, 6, 7)]

futile.Utils.split_kwargs(kwargs, sub_kwargs)[source]

Split a dictionary of kwargs from a subset.

  • kwargs (dict) – the original dictionary.

  • sub_kwargs (dict) – dictionary providing the {‘key’: default} set of arguments which should be splitted from kwargs.


(new_kw, sub_kw) the splitted dictionaries

Return type



Checks the coherency of the tarfile.


filename (str) – path of the tarfile.


True if the tarfile is good to go.

Return type



Exception if the tarfile is not valid.


Provides a filename with a unique id appended


prefix (str) – the prefix of the file



Return type


futile.Utils.unpack_tarball(archive, tmpdir_prefix='tmp_')[source]

Open an archive in a temporary directory

  • archive (str) – the path of th archive to open

  • tmpdir_prefix (str) – prefix of the temporary directory to untar the archive to.


tmpdir, files path of the temporary directory and names of

the files extracted by the tarfile

Return type


futile.Utils.untar_archive(archive, dest='.')[source]

Untar the archive in the destination directory.

  • archive (str) – path of the file to untar.

  • dest (str) – destination directory. Create if not exists.


list of the files contained in the tarball.

Return type


futile.Utils.version_is_compatible(desired_version, present_version)[source]

Assess the compatibility of a version id with a reference.

  • desired_version (str) – the version which is used as the reference, in the format “x.y”, “x.y.z”, “x.y.z.w”, …

  • present_version (str) – the version to be tested against.


True if all the version numbers of the present version are

of lower number than the desired one.

Return type


futile.Utils.write(*args, **kwargs)[source]

Wrapper for print function or print to ensure compatibility with python 2 The arguments are used similarly as the print_function They can also be generalized to python 2 cases

YamlIO module

Inputvars module

Handle the input variable specifications.

This module is the python complement to the specification of the input variables of a file as provided by the f_input_file module. It uses the same syntax as defined there and make possible the interplay between a python-based serialization of the input dictionary and the construction of

class futile.Inputvars.InputVariable(name, spec)[source]

Bases: object

Define a input variable of a library. Such object can be initialized and inspected from the dictionary used by futile and then initialized according to the provided specification.


Set the value of the input variable


Set the dependent variable from which impose the profile


Set the variable which is activated when the present has suitable values or profiles

ArgParse module

Handle and automatize the parsing of input arguments

This module uses the same convention of the yaml_parse fortran module to define the command line arguments. Such module can be used to define command line arguments of python scripts that follows the same conventions or to generate python functions that have the same signature than the provided command arguments.

futile.YamlArgparse.get_python_function(target_kwargs_function, func_name, func_spec)[source]

Convert a argparse spec into a python function

This function provides a python function with a signature indicated by the fun_spec dictionary With the conventions of the yaml_argparse modules. The futile.Utils.function_signature_regenerator() function is used for the conversion

  • target_kwargs_function (func) – the keyword arguments function we want to give the signature to.

  • func_name (str) – Name of the function, usually the key of the dictionary whose func_spec is the value

  • func_spec (dict) – dictionary of the function specifications to be provided to the futile.Utils.function_signature_regenerator() function.


the genreated function with signature given by the arguments of func_spec

defaulting to their default value.

Return type



Create the docstring of the generated function by also including the docstring of the arguments

Figures module

class futile.Figures.AxisSet(fig, rect, *, facecolor=None, frameon=True, sharex=None, sharey=None, label='', xscale=None, yscale=None, box_aspect=None, **kwargs)[source]

Bases: matplotlib.axes._axes.Axes


Clear the Axes.

set(*, adjustable=<UNSET>, agg_filter=<UNSET>, alpha=<UNSET>, anchor=<UNSET>, animated=<UNSET>, aspect=<UNSET>, autoscale_on=<UNSET>, autoscalex_on=<UNSET>, autoscaley_on=<UNSET>, axes_locator=<UNSET>, axisbelow=<UNSET>, box_aspect=<UNSET>, clip_box=<UNSET>, clip_on=<UNSET>, clip_path=<UNSET>, facecolor=<UNSET>, frame_on=<UNSET>, gid=<UNSET>, in_layout=<UNSET>, label=<UNSET>, navigate=<UNSET>, path_effects=<UNSET>, picker=<UNSET>, position=<UNSET>, prop_cycle=<UNSET>, rasterization_zorder=<UNSET>, rasterized=<UNSET>, sketch_params=<UNSET>, snap=<UNSET>, title=<UNSET>, transform=<UNSET>, url=<UNSET>, visible=<UNSET>, xbound=<UNSET>, xlabel=<UNSET>, xlim=<UNSET>, xmargin=<UNSET>, xscale=<UNSET>, xticklabels=<UNSET>, xticks=<UNSET>, ybound=<UNSET>, ylabel=<UNSET>, ylim=<UNSET>, ymargin=<UNSET>, yscale=<UNSET>, yticklabels=<UNSET>, yticks=<UNSET>, zorder=<UNSET>)

Set multiple properties at once.

Supported properties are


adjustable: {‘box’, ‘datalim’} agg_filter: a filter function, which takes a (m, n, 3) float array and a dpi value, and returns a (m, n, 3) array alpha: scalar or None anchor: (float, float) or {‘C’, ‘SW’, ‘S’, ‘SE’, ‘E’, ‘NE’, …} animated: bool aspect: {‘auto’, ‘equal’} or float autoscale_on: bool autoscalex_on: bool autoscaley_on: bool axes_locator: Callable[[Axes, Renderer], Bbox] axisbelow: bool or ‘line’ box_aspect: float or None clip_box: .Bbox clip_on: bool clip_path: Patch or (Path, Transform) or None facecolor or fc: color figure: .Figure frame_on: bool gid: str in_layout: bool label: object navigate: bool navigate_mode: unknown path_effects: .AbstractPathEffect picker: None or bool or float or callable position: [left, bottom, width, height] or ~matplotlib.transforms.Bbox prop_cycle: unknown rasterization_zorder: float or None rasterized: bool sketch_params: (scale: float, length: float, randomness: float) snap: bool or None title: str transform: .Transform url: str visible: bool xbound: unknown xlabel: str xlim: (bottom: float, top: float) xmargin: float greater than -0.5 xscale: {“linear”, “log”, “symlog”, “logit”, …} or .ScaleBase xticklabels: unknown xticks: unknown ybound: unknown ylabel: str ylim: (bottom: float, top: float) ymargin: float greater than -0.5 yscale: {“linear”, “log”, “symlog”, “logit”, …} or .ScaleBase yticklabels: unknown yticks: unknown zorder: float

classmethod twinify(ax)[source]

Include the axis provided as a radix of the set

class futile.Figures.FigureSet(**kwargs)[source]

Bases: object

Container for multiple figures.

Define a container for a plot with the possiblity to switch between simple and gnuplot plotting

Arguments: title: The title of the master figure **kwargs: arguments for the axis instance


True if the Figure exists in the Set

class futile.Figures.VertSlider(ax, label, valmin, valmax, valinit=0.5, valfmt='%1.2f', closedmin=True, closedmax=True, slidermin=None, slidermax=None, dragging=True, **kwargs)[source]

Bases: matplotlib.widgets.AxesWidget

A slider representing a floating point range.

For the slider to remain responsive you must maintain a reference to it.


the slider matplotlib.axes.Axes instance


the current slider value


a matplotlib.lines.Line2D instance representing the initial value of the slider


A matplotlib.patches.Polygon instance which is the slider knob


the format string for formatting the slider text


a matplotlib.text.Text instance for the slider label


whether the slider is closed on the minimum


whether the slider is closed on the maximum


another slider - if not None, this slider must be greater than slidermin


another slider - if not None, this slider must be less than slidermax


allow for mouse dragging on slider

Call on_changed() to connect to the slider event


remove the observer with connection id cid


When the slider value is changed, call func with the new slider position

A connection id is returned which can be used to disconnect


reset the slider to the initial value if needed

futile.Figures.axis_from_data(fig, ax, data)[source]

Transform a data tuple into axis coordinates

futile.Figures.data_from_data(fig, dst, src, data)[source]

Transform a data tuple of anothe axis in the figure into data of another axis

futile.Figures.show_image(imgfile, title=None)[source]

Show image file using matplotlib imgread. Useful to bypass the Jupyter bug for converting a notebook into a pdf file

Time module

class futile.Time.TimeData(*filenames, **kwargs)[source]

Bases: object

barwidth = 0.9
collect_categories(dict_list, vals)[source]

Collect all the categories which belong to all the dictionaries


Inspect the available counters

draw_barfigure(fig, axis, data, title)[source]
draw_lineplot(ax, data, label)[source]
find_items(category, dict_list)[source]

For a given category find the items which have them


Determine lookup array of unbalanced categories

gnuplot_figure(lookup=None, epsfile=None, aggregate=None, select_category=None)[source]

Create a figure to be plotted with gnuplot

Create a gnuplot histogram that can be plotted for production results.

  • lookup (list) – A list of the items of the TimeData instance that will be considered for the histogram

  • epsfile (str) – Name of the eps file in which the data will be plot

  • aggregate (list) – list of tuples of the categories to aggregate, accoding to the convention (newkey ,[list of oldkeys]) between the original keys and the desired keys

  • select_category (str) – Name of the category to be plot, if specified the internal items of the category will be plot.

ignored_counters = ['CPU parallelism', 'Routines timing and number of calls', 'SUMMARY', 'Report timestamp', 'Hostnames']
inspect_counter(counter, unit=None)[source]
load_unbalancing(ax, dict, hosts)[source]

Extract the data for plotting the hostname balancings between different categories in bar chart

routines_plot(index, event=None)[source]

Draw the plot of the routines level for the run identified by index


Criterion for unbalancing

workload_plot(index, event=None)[source]

Draw the plot of the workload of different classes for the run identified by index

futile.Time.aggregate_names(data, list_agg)[source]

Aggregate the names of the plot in some different categories the structure of dict_agg should ba a list of tuples (newkey ,[list of oldkeys]) between the original keys and the desired keys

class futile.Time.polar_axis(fig, ax, data)[source]

Bases: object

dump_timing_level(level, starting_point=None, ilev=0, theta=0, data=None)[source]

Inspect the first level of the given dictionary and dump the profile subroutines at this level

step = 5